Often professionals author reports or documents to convey useful information so that other professionals can take action. Sometimes hundreds or thousands of reports may be generated by a single expert during the course of a year. For example, a radiologist interprets a medical image, and produces a short one page report which conveys diagnostic findings and conclusions to a referring physician. The quality of the report is crucial to patient diagnosis and treatment. Even infrequent lapses in report quality can affect the lives of many individuals since a typical radiologist dictates 18,000 reports per year. Unfortunately, radiology reports, like many other types of medical documents are non-standardized and frequently vague, incomplete and error prone [Johnson A J, Radiology Report Quality: A Cohort Study of Point-and-Click Structured Reporting versus Conventional Dictation. Academic Radiology 2002; 9:1056-1061.]
Although some guidelines are available to construct reports and documents of different types, rarely are those guidelines codified in a form that can be displayed in a context specific manner. Templates have been used to improve data collection by reducing missing, incorrect, and inconsistent data. Templates are outlines that structure text into blocks, paragraphs, or sentences. Frequently they contain well defined delimiters that are meant to be completed (instantiated) to produce a final document. Some physicians have used templates to improve report quality since a number of important observations may be embedded in a template.
However, in medicine as well as other complex domains, specialized knowledge must be conveyed, which is not stereotyped or easily encoded in a few master templates. There are simply too many different document types and context specific conditions, which vary from case to case to make template use practical. For example, a radiology report for a normal head computed tomography (CT) scan, is different from a normal sinus x-ray exam, which in turn is vastly different than a head CT with evidence of intraparenchymal hemorrhage.
Expert Critiquing Systems (ECSs) have been used in a variety of situations to provide a broad array of supportive functions. One appeal of such systems [Silverman B G. Survey of Expert Critiquing Systems:
Practical and Theoretical Frontiers. Communications of the Association of Computing Machinery, Volume 35(4): 106-127: 1992] is to complement problem-solving systems through useful “critiques.” In medical expert critiquing systems, a physician typically defines his/her treatment plan, after which the ECS performs a separate analysis. The ECS then calculates a “difference” between the proposed and computed solution, and highlights weaknesses in the proposed treatment plan. These rule-based expert systems are generally aimed at providing an analysis of a proposed treatment, rather than pertinent questions to further explore the problem space or refine document content. One drawback of “critiques” is professionals are more likely to modify a plan or a report by considering specific questions rather than simply being told what is wrong. Additionally, medical ECSs often give their critiques after all the input is known, and are thus of limited value in real-time document creation.
Nevertheless, ECSs have aided writers through grammar and spelling critiques. Domini, et al. (Method and system for verifying accuracy of spelling and grammatical composition of a document, U.S. Pat. No. 6,085,206) disclosed a system for verifying the accuracy of the grammatical composition of sentences within electronic documents. The real-time display of pertinent corrections provided a way to improve the grammatical aspects of document quality. However, similar systems have not been used to generate additional content suggestions or questions which may improve more fundament aspects of document quality.
Commercial structured documentation applications exist that can produce coded input to an expert system, and with additional engineering could be designed to generate questions that could improve report quality. [Langlotz C P. Enhancing the expressiveness of structured reporting systems. J Digit Imaging 2000 May; 13 (2 Supplement 1):49-53.] However, these systems restrict users input to a predefined number of “codeable” entries, which in complex domains such as medicine, fall far short of the expressiveness of natural language required by their target audience.
One major hurdle to providing context specific questions, which could improve document content, is having a detailed semantic understanding of the language used by document authors. Semantic extraction starts with defining the relevant concepts in a circumscribed area of knowledge—a domain. Few tools and methods are available for systematically categorizing domain knowledge, especially for medium to large scale domain. Knowledge engineers often spend months creating even modest knowledge bases. Without a semantic knowledge-base, it is impossible to trigger context specific questions.
Another hurdle that must be overcome to make the questions helpful to authors, is a control system that prioritizes the most important questions that should be considered, and suppresses questions that have already been answered by the information in the document.
Finally, the cost of building an automated advisory system is currently prohibitive because there are no tools to create the questions or control under which context they will appear.